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A  COLUMN  GENERATION  TECHNIQUE  FOR  THE 
COMPUTATION  OF  STATIONARY  POINTS 

Jong-Shi  pang 

A3S TRAC T .  In  :vo  recent  papers,  Eaves  shoved  that  Lemke ' s  algorithm 

can  be  used  to  compute  a  stationary  point  of  an  affine  function  over  a 
polyhedral  set.  This  paper  proposes  an  alternative  method  which  is 
based  on  parametric  principal  pivoting.  The  proposed  method  involves 
solving  systems  of  linear  equations  and  parametric  linear  subprograms 
over  the  given  polyhedral  set.  An  obvious  advantage  of  the  method  is 
that  any  special  structure  of  the  polyhedral  set  can  be  exploited 
profitably  in  the  solution  of  che  subprograms. 


1. 


INTRODUCTION' 


Given  a  polyhedral  sec 

X  -  fx  €  R  :  Cx  >  c 

v  m  4 

and  an  affine  mapping 

F(x)  -  b  +  Ax 
n  n 

from  R  inco  R  ,  the  stationary  point  problem  is  to  find  a  vector  u€X 

such  chat  the  condition  below  is  satisfied 

(t.  1)  (x  -  u)T  F(u)  >  0  for  all  x  €X  . 

Such  a  vector  u  is  called  a  stationary  point. 

As  pointed  out  by  Eaves  [3],  the  stationary  point  problem  is  central 
to  the  solution  of  certain  quadratic  programs,  matrix  games  and  economic 
equilibrium.  An  important  special  case  of  the  problem  is  where  the 
sec  X  is  a  polyhedral  cone.  In  this  case,  it  has  been  shown  (see  [6] 
e.g.  )  that  the  stationary  point  problem  is  equivalent  to  the 
generalized  linear  complementarity  problem:  find  u  €.X  such  that 

F(u>  € X*  and  u^u)  -  0 
where  X*  is  the  dual  cone  of  X,  l.e., 

X*  -  {  y  €  Rn  :  yTx  >  0  for  all  x€x]  . 

In  the  reference,  Eaves  showed  that  Lemke's  algorithm  can  be  used 
to  solve  the  stationary  point  problem.  His  method  of  analysts  can  be 
briefly  sussnarized  as  follows.  First  observe  chat  the  stationary  point 
problem  (1.1)  is  equivalent  to  finding  vectors  u,  s  and  t  such  that 


■> 


T 

Cu  -  s  ■  c  ,  b+Au+Ct-0 
T 

s ,  c  >  0  and  s  c  ■  0  . 

He  then  adjoins  an  additional  sec  of  constraints  Bx  v.  a  so  Chat  the 
sys  cem 

Cx  >  c  ,  Bx  <  a 

m  m 

has  a  unique  solucion  in  the  polyhedron  X.  Next,  he  applies  complementary 
pivoting  to  the  augmented  system 

Cu-s  ■  c  b-»-Au+  CTc  +  B^*w  ■  0 

T  T 

3u+y-e9«a  s  t  •  y  w  -  0 

s>0,t>0,y>0  ,  w  >  0 

starting  with  9  equal  to  zero  and  increasing  9  to  infinity.  In  a  finite 
number  of  pivots,  the  algorithm  terminates  either  on  a  ray  or  with  a 
desired  stationer;/  point  to  the  given  problem.  Basically,  no  specific 

assumption  is  needed  to  operate  the  algorithm. 

In  a  related  paper  (4),  Eaves  describes  another  way  to  start  Lemke’s 

algorithm  for  solving  the  same  problem. 

Our  purpose  in  the  present  paper  is  to  propose  an  alternative 
approach  for  solving  the  stationary  point  problem.  the  approach  is  based 
on  parametric  principal  pivoting,  the  ideas  involved  are  briefly  sketched 
as  follows.  By  using  the  representation  of  X  in  terms  of  its  extreme 
points  and  rays,  it  is  first  shown  that  the  stationary  point  problem  can 
be  converted  into  an  equivalent  linear  complementarity  problem.  Under 
a  certain  positive  semi -definiteness  assumption  on  the  matrix  A,  the 
resulting  linear  complementarity  problem  has  a  positive  semi -definite 
matrix.  Consequently,  the  parametric  principal  pivoting  algorithm  [2.  7]  is 
applicable.  The  application,  however,  is  crucially  dependent  on  the 


knowledge  of  the  sec  of  extreme  points  and  rays  of  X.  A.s  the  task  of 
generating  all  these  points  and  rays  is  practically  isspossible,  it  is 
therefore  important  to  be  able  to  Implement  the  algorithm  without  the 
full  knowledge  of  the  generators.  By  means  of  a  column  generation 
technique  similar  to  the  one  in  linear  programing ,  we  shall  show  how 
the  useful  components  can  be  generated  when  they  are  needed,  thereby 
establishing  the  applicability  of  the  parametric  principal  pivoting 
algorithm  for  solving  the  stationary  point  problem.  We  shall  also  discuss 
a  considerably  simplified  version  of  this  algorithm  applicable  to  the 
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In  ocher  words,  the  stationary  point  problem  can  always  be  cast,  theoretically, 
as  an  ordinary  linear  complementarity  problem  of  the  form 


(2.3) 

w  -  q  +  Mi 

>  0  ,  z  >  0  and 

wT:  -  0 

where 

the  vector  q  and  matrix  M  are  given  by 

(2.4) 

<?  -  /  ?Tb  \ 

M  -  i  P‘A? 

ptaq 

-e 

e  \ 

1  QTb  < 

1  Q'ap 

QTaq 

0 

' 

0 

1  T 

•’i 

\  e 

0 

0 

0 

\  J 

\  -eT 

0 

0 

0  1 

It  should  be  pointed  out  that  the  derivation  of  this  equivalent 
linear  complementarity  problem  does  not  require  any  assumption  on  the 
set  X  and  the  matrix  A. 


3. 


THE  PARAMETRIC  PRINCIPAL  PIVOTING  ALGORITHM 


Throughout  chi.  paper,  we  assume  that  the  matrix  A  is  positive  semi- 
definite  over  the  linear  subspace  spanned  by  the  sec  X.  (This  assumption 
is  much  weaker  chan  that  of  a  positive  semi -definite  A.)  It  is  not 
difficult  to  show  that  this  assumption  is  equivalent  to  the  fact  that  the 
matrix 


1  ?‘A? 

( 

T 

P  AQ 

V  Q~ap 

qtaq 

A  (P  Q) 


is  positive  semi -definite.  Thus,  so  is  the  matrix  M  in  (2. A).  In 
particular,  the  parametric  principal  pivoting  algorithm  [2,  7]  can  be  used 
to  solve  the  linear  complementarity  problem  (2.2).  However,  this  approach 
Is  certainly  ineffective  if  it  is  necessjry  to  know  the  whole  matrices  P 
and  Q.  To  demonstrate  how  the  algorithm  can  be  implemented  without  the 
full  knowledge  of  these  matrices  of  generators,  we  first  state  a  version 
of  Che  algorithm  which  operates  by  updating  the  constant  and  parametric 
columns  only. 


The  parametric  Principal  Pivoting  Algorithm.. 

Step  0  (Initialization)  Let  J  -  (9  and  let  I  be  the  complement  of  J. 

Seen  1  (Computing  the  basic  components)  Solve  the  system  of  linear  equations— 
for  (q j ,  Pj): 

(3.1.)  .  Pj)  -  -  (qj  .  Pj)  • 

1/  If  M  is  a  matrix  and  K  and  L  are  index  sets,  by  we  mean  the  subratrix 

of  M  whose  columns  and  rows  are  Indexed  by  K  and  L  respectively.  A 
similar  notation  is  used  for  vectors. 
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See?  2  (Computing  the  nonbasic  components)  Compute 

(3* Tb)  (v  pr>  •  (V  px>  +  Vv  Pj> 


Step  3  (Ratio  test)  If  the  vector  p 
with  the  solution 


is  nonpositive,  terminate 


(3.:) 


z*  - 


/ 


to  the  linear  complementarity  problem  (2.3) 


Otherwise  determine 


(30)  9 ij  /  Pj  = 

and  let  k  be  a  maximizing  index.  If  a  v  0,  terminate  with  the  solution 
given  by  (3.2).  If  a  >  0,  continue. 

Step  a  (Check  pivot  element)  There  are  two  cases 

(I)  k€J.  Solve  the  system  of  linear  equations  for  f  : 

(3. -a)  Mjjfj  -  eK 

where  e*  is  a  unit  vector  with  a  one  in  component  k.  Compute 

(3.4b)  «j  •  Mj/j  • 

(II)  k£j.  Solve  the  system  of  linear  equations  for  f,  : 

c  a  .  u 

JJ‘J  Jk 


(3.5a)  M 
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and  coapuce 


(3.5b)  f,  . 

In  either  case,  if  £,^  ■  0  go  to  Step  t>.  Otherwise  continue. 


Step  5  (1  x  1  diagonal  pivot)  Sec 


j  -  r  j  , ,  u  f  w  ] 

new  old  *  J 


if  k  £  J 


old 


! J  . .  \  (k) 

k.  old 


otherwi se . 


Go  to  Step  1 


Step  p  (2x2  block  pivot1)  If  the  vector  f 


(!) 


is  nonnecative , 


stop,  che  coap Leaentari ty  problea  (2.3)  is  infeasible.  Otherwise 
deteraine  another  index  i  bv 


(3.6) 


-  (q,  + 

aV '  *i 

-  sin  {  -  (q. 

+  9Pi)/fi  : 

fi  <  ol 

coaputed 

in  (3.3). 

Set 

Jr.ev  ' 

|  Jold 

4} 

if  k,  4€JoU 

j 

Jold  ' 

W  u  {4} 

if  k  €  J  ,  ,  and 
old 

4fJ  .. 

r  old 

Jold 

[l]  U  tk) 

if  K  €  J  .  .  and  k  {  J 

old  •  old 

^  Jold  •’ 

*>,  4} 

otherwise  . 

Go  to  Step  l. 

In  the  description  of  the  algoritnm  above,  J  is  the  index  set  of  the 
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basic  z -variables  (cf.  (2.3)),  ?  is  the  nonnegative  parameter  to  be  driven 

★ 

to  zero  and  p  is  any  vector  satisfying  the  condition:  q  +  9  p  >  0  for 
some  3  >0. 

Rigorously  speaking,  cycling  could  occur  in  the  algorithm.  Often 
this  can  be  prevented  by  a  lexicographic  or  least-index  rule  [1]. 

According  to  a  basic  result  in  pivotal  algebra,  the  "basis  matrix" 

Mjj  is  nonsingular  throughout  the  algorithm.  Referring  to  the  linear 
cemplementari ty  problem  (2.2),  the  nons irgulari ty  of  T  implies,  among 
other  things,  that  the  numbers  of  basic  "  -  and  5‘variabies  are  bounded 
by  n  +  1  and  n  respectively.  (Recall  that  n  is  the  order  of  the  matrix  A.) 
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nt£  jgrrEK,-. ::o::  ^c;i:;i'.e 

In  this  section,  we  shew  how  the  parametric  principal  pivoting 
algorithm  described  in  the  -cc:  section  can  be  operated  if  the  matrices  P 
and  Q  are  known  only  implicitly. 

As  pointed  out  at  the  end  of  the  last  section,  at  each  iteration, 
there  Is  a  reasonable  limit  of  Information  pertaining  to  the  basic  components. 
Consequently  they  can  be  stored  without  difficulty.  However,  it  is 
necessary  to  derive  an  alternate  way  to  handle  the  nonbasic  components. 

From  the  description  of  the  algorithm,  it  is  obvious  that  these  components 
are  needed  for  the  determination  of  the  indices  k  and  i  and  the  corresponding 
columns  M„  and  M,,.  As  a  matter  of  fact,  the  latter  indices  and  columns 
arc  the  key  substances  that  one  wishes  to  obtain  at  the  end  of  the  iteration. 
Consequently,  if  they  can  be  determined  without  the  complete  knowledge  of 
the  nonbasic  components,  then  one  can  operate  the  algorithm  readily.  In 
what  follows,  ve  demonstrate  how  this  important  step  can  be  accomplished 
by  means  of  a  col’-san  generation  technique. 

3efore  starting,  we  would  like  to  say  a  few  words  about  the  choice 

of  the  parametric  vector  p.  In  principle,  it  can  be  any  vector  satisfying 

♦  * 

the  ondition  that  q  -*-  ?  p  is  nonnegative  for  some  nonnegative  §  .  The 

latter  condition  ensures  a  valid  start  of  the  algorithm.  However,  in  the 
present  situation  where  the  vector  q  is  known  only  implicitly,  the  choice 
of  p  turns  out  to  be  quite  crucial.  This  will  become  obvious  in  the 
analysis  below. 

U.y.  The  3ounded  Case.  We  divide  our  discussion  into  two  cases 


depending  on  whether  the  «et  X  is  bounded.  We  first  consider  the  case 


where  the  X  is  bounded.  In  this  case,  the  linear  complementarity  problem 
(J.2)  reduces  to 


(-.  1) 


T  T 

1  b  ♦  P‘AP* 

-  /+e  X  e  >0 

-  >  0 

■ 

■ 

-1  ♦  eT“ 

>  0 

\+>  0 

■ 

T 

1  -  e*~ 

>  0 

O 

Al 

,< 

(  ?.  )  V  “  OlV  |i’  -  0 


For  the  parametric  vector ,  we  choose  p  *  (e‘,  I,  0)‘  . 

Fecal  I  that  the  objective  for  the  ratio  test  in  Step  3  is  to  determine 

a  new  critical  value  of  the  parameter  ?  such  that  vector  q  -  p  (whose 

c  opponent  s  give  the  updated  values  of  the  currently  basic  variables)  will 

remain  nonnegative.  Let  K  denote  the  index  set  of  the  currently  basic 

“-variables.  To  xeep  the  notations  simple,  we  assume  that  both  \*  and  \ 

♦  • 

are  nonbaslc.  ^Notice  that  \  and  \  cannot  be  simultaneously  basic.) 

The  basis  matrix  is 


V  •  'V  apk 


where  P„  denotes  the  columns  of  ?  indexed  by  K.  According  to  (3.1b),  the 
nonbasic  components  are  given  by 

<q..P,)-/<P  ~)Tb  e-  \  +  l(?-V  AP„\  <q..  P„) 


V 


where  K  denotes  the  complement  of  K  and  P7  the  columns  of  P  indexed  by 

K 

K  .  Hence,  (q  *  -p)  is  nonnegative  if  and  only  if 


<*  .2a) 


(P-  )T(b*A  PKqK  ♦  APkPk  ]  ^  -  re- 


<i-21”  ■'•'iV'i'-iv:' 

“•2°  '-•k\-,v>k  i° 


Among  these  three  conditions  only  the  first  one  (4.2a)  requires  special 
attention  because  it  involves  the  (unknown'  matrix  P  -  .  Obviously,  it 
folios  from  (3.1a)  that  for  all  values  of 


(4.3,  (p/|b.APK^4-*PK^!  - 

Thus  (4.2a)  holds  if  and  only  if 

(P)4  [b  -  A  Pj.  *  iA?).  Pj.  )  ^  -  -  e 

or  equivalently 

T  . 

x 1  [ b  A  P  q  ♦-*?.?!  >  -  -1  for  all  x  c  X 

K  K  K  K  * 

Observe  how  the  parametric  vector  is  being  used  to  derive  this  last 
i  rse  q  ua  1 1 1  y . 

Consequent ly  to  determine  the  first  value  of  -  for  which  (4.2a)  is 
violated,  one  may  proceed  as  follcvs:  Solve  the  parametric  linear  program 

(4.4)  minimi  re  L^(a)  •  x^I  b  +  A  P„  q  R  -  A  P^.  p^]  subject  to  xcX 

starting  with  the  last  critical  value  of  for  which  the  Inequality  be  1  rsi  is 
satisfied 


-.5 


l  (*>  *JjO 


’  3 


and  decreasing  Che  parameter  until  either  it  reaches  below  zero  or  a 

value  -  and  a  correspond ing  vector  P  -  is  obtained  for  which  the 

1  1 

inequality  (4.5)  is  violated.  For  notatlonal  convenience,  we  let  -  be 

zero  if  -  reaches  zero  before  the  violation  of  the  inequality.  Notice 

that  the  vector  P.  is  an  optimal  solution  to  the  linear  program  (4.4) 

*1 

1  1  1 
with  .  It  is  important  to  point  out  that  if  i  is  positive,  the 

index  obtained  must  be  in  K  because  equality  in  (4.3)  holds  for  all 

values  of  - . 

With  Che  value  ^  available,  the  ratio  test  (3.3)  can  now  be  implemented 
as  follows:  Determine 


1£  *k’k< 0 


■  <_l  *  *<V/(1  **KPK>  lf  *1<#kPkS° 


otherwise 


and  then  set 

12  3 

(*•.6)  -  -  max— 

This  final  value  would  give  the  desired  new  critical  value.  The  maxi¬ 
mizing  index  it  can  be  determined  in  an  obvious  manner. 

<+• 

F.ssentially ,  the  same  analysis  can  be  applied  lf  either  X  or  \  is 
basic.  Orta  may  still  implement  the  ratio  test  (3.3)  by  solving  the  same 
parametric  linear  program  and  by  comparing  the  ratios  among  the  basic 
components.  A  noteworthy  point  is  that  the  inequality  to  be  checked  in 
this  instance  would  be  slightly  different  frcxa  (4.5),  h»ever. 


Having  performed  the  first  ratio  test,  we  proceed  to  show  how  the 


second  one  <3. 6)  can  be  carried  out  without  the  explicit  kncvledge  of 
the  whole  matrix  ?.  3efore  doing  this,  we  point  out  that  the  column 
>lfk  re<*uireJ  (3.5a)  is  easy  to  generate  once  the  maximizing  index  k 
is  determined. 

Recall  that  the  ratio  test  in  (3.6)  is  to  determine  the  largest  value 
o  >0  so  that  the  vector  (q  +  ;  p )  +  o  f  remains  nonnegative.  Here  j 
denotes  that  variable  correspond ing  to  the  index  k.  To  be  specific 
we  assume  that  '<  is  equal  to  the  index  k,  introduced  earlier.  TTien. 

by  (3.1b)  and  (3.5b)  we  have 


(c  +  •  p) 


‘"A"11*'!  V*  5('*'k  V 


T  -  T  •  ^ 

1  •  e  *  Q  -  «  «* 4  p  -  j  ( 1  +  e  4  f  ) 
K^K  v  K  K 


By  using  a  similar  analysis  and  by  noting  that  the  equality  helow  is 
valid  for  all  values  of  r,  namely 

,Vt'4*aVVaa^*:'ap1*apkVi<-?'k-  ’  • 

one  may  easily  obtain  the  minimum  ratio  (3.6)  in  the  following  manner: 

Solve  the  parametric  linear  program 

<u.7)  minimize  L‘(s)-x  f  b  *  AP..q  R  a-  *  c  (AP.^  *  ApK.f  ^  ))  subject  to  x  €  X 

starting  with  the  value  c  "0  for  which  the  inequality  below  is  satisfied 


(4.8) 


L  (c)  +  ~  >  0 
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ami  increasing  the  parameter 


until  (4.8)  is  violated.  Let  ;  Lo  the 


first  such  value  obtained  and  let  P^  be  the  corresponding  optimal 
solution  vector.  If  (4.3)  remains  satisfied  for  all  values  of  p,  set  : 
Next,  determine 


1 


l€K' 


C  -  aln'-(q1  •*-&p1)/f1  :  <  0 


if  1  +#^f  •>  0 


otherw ise 


and  then  set 


1  2  3. 


The  desired  index  l  can  new  be  determined  in  an  obvious  manner. 

Again,  the  same  analysis  can  be  applied  if  the  maximum  in  (4.6) 

3  ♦ 

occurs  at  -  ,  l.e.,  if  either  \  or  \  is  becoming  basic.  In  this  case, 
the  parametric  linear  program  (4.7)  and  the  inequality  (4.8)  would  be 
slightly  changed  but  the  essential  idea  would  remain  unchanged. 

We  point  out  that  if  K  is  the  final  index  set  of  basic  ''-variables 
obtained  at  termination  of  the  algorithm,  then  a  solution  to  the  stationary 
point  problem  is  given  by  which  can  be  computed  easily. 

4.2  The  Positive  Definite  Case.  A  considerably  simplified  version  of  the 
parametric  principal  pivoting  algorithm  can  be  used  if  the  set  X  is 
bounded  and  *f  the  matrix  A  is  positive  definite.  The  simplification  stems 
from  the  fact  that  the  linear  complementarity  problem  (4.1)  has  an  obvious 


■ 
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reformulation  under  which  the  algorithm  will  never  perform  the  2x2  pivots. 
In  particular,  only  one  parametric  linear  program  needs  to  be  solved. 

It  is  obvious  that  the  linear  complementarity  problem  (4.1)  is 
equivalent  to  the  following  one:  find  X  and  "  such  that 


(i.9a) 


C  -  PTb  +  Xe  +  P ; AP“  >  0 


-  >  0 


(-.9b) 


1  *  e 


and  X  unrestricted. 


Condition  (4.9a1  constitutes  a  parametric  linear  complementarity 
problem  (with  X  as  the  parameter)  to  which  the  parametric  principal  pivoting 
algorithm  is  applicable.  Concerning  this  application,  we  have  the  following 
result  whose  proof  car.  be  found  in  the  reference  cited. 

Theorem  '  .  (Kar.eko  and  Pang  (5])  Suppose  that  the  matrix  A.  is  positive 
definite.  Consider  the  application  of  the  parametric  principal  pivoting 
algorithm  to  the  parametric  linear  complementarity  problem  (4.9a).  Then 
each  diagonal  pivot  entry  f,^  (cf.  Step  4  of  the  algorithm)  is  positive  and 
thus  the  2x2  block  pivots  are  redundant.  In  particular,  the  algorithm  will 
always  compute  a  solution  to  the  problem  for  all  values  of  X.  Let  ~*(X) 
denote  the  solution  obtained  as  a  function  of  X. 

It  is  Important  to  point  out  that  for  each  X,  ”*(X)  may  not  be  the 

only  solution  to  the  linear  complementarity  problem  (4.9a).  In  the  rest 
of  this  subsection,  we  assume  that  A  is  positive  definite. 

Taking  into  consideration  the  condition  (4.9b),  we  have 
Theorem  2.  Suppose  that  problem  (4.9)  has  a  solution  (",  X)  with 


X  4  0.  Then  each  solution  "*  to  the  linear  complementarity  problem  (4.9a) 
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with  X  »  \  also  satisfies  (4.9b).  In  particular,  so  does  the  solution 
~*(\)  obtained  in  Theorem  1 . 

Proof .  Subtracting  the  rvo  equations 

C*  -  PTb  +  Xe  +  PTAP-*  ,  ~  -  PTb  +  Xe  +  PTAP- 

we  obtain 

C*  -  C  -  PlAP(-*  -  -) 

which  implies 

o  >  r*  -  i>T(c*  -  6)  -  (?r*  *  7>)T  apc*  -  -j)  >  o  . 

3y  the  positive  definiteness  of  A.  it  follows  that 

PC*  -  -  0  . 

Consequently,  we  have 

0  -  (~*)T  c*  "  (P"*)1  b  +  X  el“*  +  (P“*)T  AP-* 

-  X  (e  -  eT~)  -  X(e7-*  -  1)  . 

3y  assumption,  X  A  0;  hence  the  desired  conclusion  follows.  Q.E.D. 

Now,  if  the  parametric  principal  pivoting  algorithm  is  applied  to 
the  parametric  linear  complementarity  problem  (4.9a),  then  either  a  value  X 
can  be  found  for  which  the  corresponding  solution  ”*(X)  satisfies  the 
condition  (4.9b)  as  well,  or  no  such  value  exists.  In  the  first  case,  a 
solution  to  the  stationary  point  problem  is  obtained.  In  the  second  case. 


there  are  two  possibilities:  either  the  problem  (4.9)  has  a  solution 
with  \  ■  0  or  it  has  no  solution  at  all.  To  determine  this,  we  may  solve 
the  system  of  linear  inequalities: 

(4.10)  ?~  -  p-*(0)  ,  e‘"  -  1  and  “  >  0. 

Theorem  3.  Suppose  that  the  above  version  of  the  parametric  principal 
pivoting  algorithm  does  not  compute  a  solution  to  the  problem  (4.9).  If 
the  system  (4.10)  is  inconsistent,  then  problem  (4.9)(  and  therefore  the 
stationary  point  problem')  has  no  solution.  On  the  other  hand,  if  **  is  any 
solution  to  (4.10),  then  (“ ,  0)  solves  (4.9). 

Proof .  3y  Theorem  2  and  the  assumption,  it  follows  that  any  solution 

(if  it  exists)  to  problem  (4.9)  must  have  X  •  0.  Suppose  that  (4.9)  does 

have  a  solution  (**,  0).  We  show  p~  •  p-*(0).  This  follows  from  a  more 

1  2 

general  result  whose  proof  is  easy  and  omitted:  If  ”  and  “  are  two 

solutions  to  the  linear  complementari ty  problem  (4.9a)  for  the  same 
1  2 

value  of  X.  then  P“  *  P~~ .  (Cf.  the  proof  of  the  last  theorem.) 
Consequently ,  if  the  system  (4.10)  is  inconsistent,  then  problem  (4.9') 
has  no  solution. 

Conversely,  let  “  be  any  solution  to  the  system  (4.10).  It  suffices 
to  show  that  (“ ,  0)  satisfies  (4.9a).  Obviously, 

PTb  +  PT\P-  -  PTb  +  PTAJ»-*(0)  >  0 

and 

;  T  (pTb  +  p^p^)  .  (**(0))T  (PTb  +  PTAP*'*(0))  -  0  . 


This  completes  the  proof  of  the  theorem. 


Q.E.D. 
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In  what  follows,  we  outline  how  the  Ideas  presented  above  can  be 
actually  carried  out  to  solve  the  stationary  point  problem  in  the  case 
where  the  matrix  ?  is  known  only  implicitly.  First,  by  the  column 
generation  technique  described  in  the  last  subsection,  the  parametric 
principal  pivoting  algorithm  applied  to  (4.9a)  can  in  fact  be  implemented. 

The  search  for  the  suitable  X  can  be  achieved  by  a  simple  interpolation 
procedure  described  in  [7  ].  The  procedure  does  not  involve  the  matrix  P  and  is 
therefore  applicable.  Finally,  the  consistency  of  the  system  (4.10)  can 
be  determined  by  checking  if  the  vector  P“*(0)  is  in  the  set  X.  Notice  that 
this  vector  is  equal  to  p  [”*(0)  }  where  K  is  the  index  set  of  basic 

K  N 

“-variables.  Since  P  is  known.  P***(0)  can  be  obtained.  Notice  also 
that  a  solution  ”  to  the  system  (4.10)  is  not  actually  needed  to  obtain 
a  solution  to  the  stationary  point  problem.  This  is  because  if  the 
vector  ?“*(0)  is  in  X,  then  it  is  a  desired  solution. 

4.3  The  Unbounded  Case.  We  extend  the  discussion  of  Section  4.1  to 
treat  the  case  where  the  set  X  is  not  necessarily  bounded. 

Associated  with  the  set  X  is  its  "homogenized'*  cone 

Y  -  {(  *  )€Rntl  :  Ca  >  ct  ,  t  >  0  }  . 

We  assume  that  the  dual  cone  Y*  contains  an  interior  point  which  is 

2/  y* 

available  explicitly . —  Such  a  point  (  *  )  will  be  used  to  define  the 

parametric  vector  p  and  is  characterized  by  the  condition: 

(y*) T x  +  s*t  >  0  for  all  (  * )  €  Y\  f  0  ]  .  In  particular,  it  holds  that 

(y*)  tP  s*  e  >  0  and  (y*)  L  Q  >  0  . 

We  shall  discuss  more  about  this  assumption  in  the  Appendix. 


:o 


Choose 

P  ■  (<y*>  *P  -*•  s*e‘  ,  (y*)TQ  ,  1  ,  0) ' 

The  reason  for  this  choice  will  became  obvious  in  a  swment.  We  should 
point  ouc  that  it  is  not  necessary  to  compute  the  vector  p  explicitly. 

In  fact,  its  components  will  be  generated  when  they  are  needed. 

Following  the  analysis  of  Section  *• .  I  ,  we  consider  a  typical  Iteration 
of  the  parametric  principal  pivoting  algorithm  applied  to  the  linear 
complementarity  problem  (2.2)  with  the  parametric  vector  p  chosen  above. 

Let  K  and  L  denote  respectively,  the  index  sets  of  the  currently  basic 
■-  and  '-variables.  This  time,  we  assume  that  is  also  basic.  The 
basts  matrix  is  then 


Partitioning  the  basic  components  in  accordance  with  the  basis  matrix,  we 
may  write 
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According  to  O.’b),  the  nonbasic  components  are  given  by 


<VV  ■  j  tFK)Tb 

(P  -  )Ty*+s*a  - 

1 

/‘V'H 

<pk,Ia1,k 

-e  -  . 

K  \ 

\ 

/i1 
;  K 

f 

- 1  i 

PK 

!  (0L)tb 

I 

(Q£  )Ty* 

"»i>H 

•  |! 

-  2 

-  2  i 

PL 

l  , 

0  / 

i  < 

0 

/  i 

°l 

• 

\  t1 
\ 

'*/ 

Hence  (q  +  9p ) ^ 

is  nonnegative  if 

and  only  if 

(4.1  U)  (P  -  )T‘*b  +A(pK\  +  +  aty*  +  A('Pk?k  +  ^  •  -  U1  +  9(t‘  -  s*)]e  - 


(4. Tib)  (Q  £)Tfb+A(P„q.’ -t-  QLq‘)*e[y*+  A(PkpJ+  Qlp‘)|  )  >  0 
(4.11c)  (1  -  'I  \  )  -  a«KpK  ’  ?  ^  ° 

Obviously,  equality  holds  for  all  values  9  in  (4.11a)  and  (4.11b)  if  K 
and  4  are  replaced  by  K  and  L  respectively.  Consequently,  (4.11a)  and 
(4.11b)  hold  if  and  only  if 

x*rb*A(PsqK’  +  QLqJ  )  +  e[y**A(PKp  j! +Qlp  “)]]  >t’  +  9(t‘-s*)  for  all  x€i  . 

Therefore ,  to  determine  the  first  value  of  9  for  which  the  condition  (4.11) 
is  violated,  one  may  proceed  as  follows :  Solve  the  parametric  linear  program 

minimize  L3(9)  -  x^b  +  +  9(y*+A(PRpf.1  -M^pJ  )]]  subject  to  x€X 


starting  with  the  last  critical  value  of  9  for  which  the  Inequality  below  is 
satisfied 

(4.1  i )  L3(9)  >t’  +  9(t‘  -  s*) 


(4.’i) 


ami  decreasing  -  until  either  it  reaches  below  zero  or  a  vlaue  9  is 
obtained  for  which  the  inequality  (4.12)  is  violated.  In  the  latter  case, 
either  an  index  k.  cR  (and  a  corresponding  optimal  solution  vector  P,  ) 

If 

1 

or  an  index  k'  cL  (and  a  corresponding  ray  vector  Q  ,  )  is  obtained  as 

*1 

well.  For  convenience,  we  let  9  be  zero  if  (4.12)  remains  valid  at  9  »  0  . 

With  the  value  9  determined  the  ratio  test  (3.3)  can  now  be  carried  out 
by  comparing  9  with  the  maximum  ratio  among  the  basic  components. 

By  a  similar  analysis,  it  can  be  shown  that  the  second  ratio  test 

(3.6)  can  be  implemented  by  solving  a  parametric  linear  program  like  the  one 

(4.7)  and  by  checking  when  a  certain  inequality  involving  the  optimum 
objective  value  and  the  parameter  of  the  program  (cf.  (4.8))  is  violated, 
the  details  are  omitted. 

Finally,  we  point  out  that  if  K  and  L  are  the  final  index  sets  of 
basic  and  '-variables,  then  a  stationary  point  is  given  by  u  -  P  *  + Q  S  • 

N  K  L  L 
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5 .  CONCLUSION 

In  this  paper,  we  have  shown  how  the  parametric  principal  pivoting 
algorithm  can  be  used  to  compute  a  stationary  point  of  an  affine  function 
over  a  polyhedral  set.  The  success  of  this  approach  depends  very  much 
on  a  proper  choice  of  the  parametric  vector  which  allows  che  algorithm  to 
operate  solely  with  the  basic  ingredients.  At  this  moment,  there  seems 
to  be  a  theoretical  drawback,  however.  Namely,  we  have  not  yet  been  able 
to  settle  the  question  of  cycling  in  Che  imp lementa tion .  One  possible 
way  to  do  this  would  be  to  extend  the  well-known  lexicographic  rule  (see 
[2)  e.g).  The  key  issue  here  is  the  question  of  how  to  incorporate  the 
rale  properly  in  the  linear  complementarity  problem  (2.2).  Special  care 
is  needed  because  the  form  of  the  constant  vector  plays  a  very  crucial 
role  in  the  column  generation  technique. 

Fortunately,  the  devil  of  cycling  has  always  been  (and  hopefully, 
will  be i  a  theoretical  concern  mainly.  So,  even  without  a  theoretically 
justified  cycling-prevention  scheme,  it  would  seem  that  che  technique 
proposed  should  still  deserve  a  try  for  solving  practical  applications. 
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appendix 


We  establish 

Theorem.  -» .  Let  S  be  a  polyhedral  cone  with  the  representations 

S  ■  'x  :  Cx  >  O'  ■  fx  :  x  -  PX  for  some  X  >  Oj  . 

where  each  generator  P  is  nonzero.  Then  the  following  are  equivalent 
(i)  The  vector  v*  is  an  interior  point  of  the  dual  cone  S* 

(il)  (y*  1  *  x  •  0  for  all  x  €  S  \  { 0 ] 

(ill)  (y*) *  P  >  0 

(iv)  For  each  vector  p,  there  exist  a  vector  X  >  0  and  a  scalar 
„  >  0  such  that 

CT  X  -  ?  *  „>•* 

(v)  For  each  vector  p,  there  exists  a  scalar  >  0  such  that 

0  •  ain  (p  4-  „y*)  *x  . 
x  €  S 

proof .  The  equivalence  of  (i),  (11),  and  (ill)  are  well-known.  That  of 
(iv)  and  (v)  Is  an  immediate  consequence  of  the  duality  theory  of  linear 
progranaing .  We  now  shew  that  (11)  Is  equivalent  to  (lv).  Obviously.  (11) 
is  equivalent  to  the  fact  that  for  all  vectors  p,  the  system  below  has 
no  solution 

T  T 

Cx  >  0  ,  -  (y*)  x  >  0  and  p  x  <  0 

3y  ttotrkin’s  theorem  of  the  alternative,  the  latter  fact  is  In  tun 


equivalent  to  that  for  each  vector  p,  there  exist  a  \  >  0  and  „  >  0  such 
that 

-  p  +  CT\  -  „y*  -  0 

which  is  precisely  condition  (iv).  Q.  E.  D. 

If  a  polyhedral  cone  is  represented  in  terns  of  its  generators, 
then  according  to  (Hi),  the  task  of  finding  an  interior  point  of  the  dual 
cone  can  be  accomplished  by  solving  a  linear  program.  On  the  other  hand , 
if  the  primal  cone  is  given  in  terns  of  a  system  of  linear  inequalities, 
then  it  may  not  be  easy  to  determine  whether  such  a  point  exists. 

However,  there  are  cases  where  this  Is  trivial.  For  instance,  if  the  given 
cone  5  lies  in  the  nonnegative  orthant,  then  the  vector  of  ones  is 
obviously  a  desired  interior  point  of  s*  by  (ii). 


t»»i  or  «e*o«T  i  •*£•« 

Technical  Report 
Oc  t  o  be  r _ 1979 


A  COLUMN  GENERATION  TECHNIQUE  FOR  THE  COMPUTATION 
OF  STATIONARY  POINTS 


I  N000I4>-  7S-C-062 1 


*  C  XtO^KWNO  ONMlttT  C  h  ««•  J 

Graduate  School  of  Industrial  Adminlstrat ion 
Carne gle -Me l Ion  University 


Pittsburgh.  Pennsylvania  15213 


o*»c«  »•«<  »*>c 

Personnel  and  Training  Research  Programs 
Office  of  Naval  Research  (Code  -»3-*) 

AtliSJ tliHi  Vft  aSllZ - 


t  Stationary  points.  Quadratic  programming ,  Linear  complementarity,  Column 
,  generation.  Principal  pivoting.  Variational  inequality 


